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Abstract 

What are the challenges faced by a mid-south liberal arts college teacher education program, in 
its attempt to successfully meet the new Council for the Accreditation of Educator Preparation 
(CAEP) accreditation requirements? The Investigator studied the Educator Program Provider 
(EPP) during academic year 2015-2016. The challenges faced by the EPP are chronicled in case 
study form. The program is one of the first in the United States preparing for the new national 
teacher education accreditation, as well as being one of the first of two in its state to experience 
the new accreditation process and requirements. The Investigator shares discussions and 
analyses, describing the challenges of assessment validity and reliability, reconfiguring program 
assessments, data collection and analysis, electronic portfolios, clinical experiences, P-12 school 
partnerships, college wide recruiting efforts, state partnerships, and quality assurance system 
revisions. The five new CAEP Standards, and their components, are compared and contrasted 
across, and within, topics. Discussion follows regarding new regulations regarding state rankings 
of teacher education programs, and entrepreneurial academies being readied to take the place of 
programs that close or fail. The case study can be useful for other teacher education programs, in 
their contemplations about how best to prepare for the challenges of CAEP accreditation. 

Keywords : CAEP, teacher education, teacher preparation, higher education, accreditation, 
Education Program Provider, EPP, validity, reliability, program assessments, program 
evaluation, data collection, data analysis, portfolios, clinical experiences, P-12 school 
partnerships, college wide recruiting, state partnerships, assessment system, quality assurance 
system, pre-service candidates, in-service candidates 
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CAEP challenges for a mid-south U.S. college teacher education program 

What are the challenges faced by a mid-south liberal arts college teacher education 
program, in its attempt to successfully meet the new Council for the Accreditation of Educator 
Preparation (CAEP) accreditation requirements? 

The Investigator studied an Educator Program Provider (EPP) during academic year 
2015-2016. The program is one of the first in the United States preparing for the new national 
teacher education accreditation, as well as being one of the first of two in its state to experience 
the new accreditation requirements and process. 

Assessment Validity 

The EPP sought national accreditation, resulting in a successful National Council for 
Accreditation of Teacher Education (NCATE) visit four years ago. The EPP also had a 
successful state accreditation visit six years ago. Areas for improvement (AFIs), resulting from 
the accreditation team visits, were identified. The AFI’s included recommendations for 
improving validity and reliability. 

Validity types include internal, external, test, content, construct, criterion, and face 
(Springer, 2010). EPPs, and their accreditors, are concerned with content, construct, and criterion 
validity. Content validity refers to how well an assessment measures all facets of what it intends 
to measure. Construct validity’s focus is on the extent to how well an assessment measures what 
it intends to measure. In other words, content validity is concerned with the merit of each of the 
components of an assessment, while construct validity is about how well an assessment measures 
what it is supposed to be measuring. Criterion validity measures the accuracy of tagging the 


assessments used to relevant state and national standards. 
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As mentioned, one of the EPPs AFIs was that the assessments being used had not been 
tested for validity. The validity issue persisted through, and including, the 2015-2016 academic 
year. 

A seminal measure used to test content validity is Lawshe’s Content Value Ratio (1975). 
The ratio is CVR= (Ne-N/2) / (N/2). The formula yields values from +1.0 to -1.0, with +1.0 
being most favorable. Experts score the components of an assessment using a 3 point scale. The 
scale items include “Essential”, “Useful, But Not Essential”, and “Not Necessary.” Simply put, if 
half or more of the experts rate the assessment component as “Essential” then the item possesses 
content validity. 

So, who are the experts that evaluate the assessments for content validity? In EPPs, the 
experts are the members of the council that is appointed by the EPP to receive recommendations 
for program changes, and who approve students being admitted into the program, student 
teaching, etc. These councils typically meet two to three times a year. 

In the EPP studied, this group is known as the Teacher Education Committee (TEC). To 
best align with the new CAEP Standards, these councils should be comprised of the EPP full 
time faculty, an adjunct representative if applicable, P-12 school district partner leaders, P-12 
school district teachers, recent graduates of the program who are teaching, and students in the 
program. 

Since EPPs have these councils, or committees, in place they can empower them to 
review the EPPs assessments, and then apply Lawshe’s ratio to the components. With CAEP’s 
emphasis on the use of rubrics in assessments, the EPP councils will need to be involved in 
evaluating validity. This will provide the needed content validity measures. 
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It should be noted that state and national assessments used by EPPs do not have to be 
tested for content validity, in that they have already been tested. Examples of these include 
AACU rubrics used to measure written work and oral presentations. Also, surveys do not need to 
be measured for validity. Surveys measure perceptions. Assessments, namely rubrics, created by 
the EPP are the focus of the content validity tests. 

Construct validity, as previously mentioned, measures the extent to which an assessment 
measures what is intended to be measured. Although it cannot be exactly determined from the 
AFI verbiage, the previous accreditation results most likely identified construct validity 
particularly as an AFI. The new division chair discovered there were such issues with an 
assessment process in one of the EPPs assessment stages, during the study year. 

Although the EPP utilized the state teacher education standards for the interview process 
for pre-service candidates seeking to enter the practicum and student teaching stage, where in 
fact a content validity test was not needed being that it was a state pre-validated instrument, it 
was the interview itself that possessed the construct validity issue. Further discussion regarding 
this issue is shared in the next portion of this paper, within the area of reconfiguring program 
assessments. 

The review of tagging assessments to standards initially revealed no issues with criterion 
validity. However, the State in which the EPP resides is one that is classified by CAEP as “State 
Review.” States, as defined by CAEP, vary among three program review options: 1) SPA 
Standards driven, 2) State program review with feedback option, and 3) State Review (CAEP 
Handbook, 2016 pp. 79-81). However, within the State’s “State Review,” SPA Standards are 
embedded. This translates as, although the state is not SPA Standards driven, SPAs must in fact 
be identified in crosswalks across the EPPs professional course syllabi, and perhaps then some. 



CAEP CHALLENGES FOR A MID-SOUTH U.S. COLLEGE 


6 


CAEP has not identified such a mash up of SPA Standards and State program review, among 
their program review options. They should do so, for the sake of clarity and guidance for 
program accreditation directors. 

Assessment Reliability 

Reliability measures the quality of procedures used in scoring assessments, and indicates 
the degree of accuracy of that scoring (Springer, 2010). Raters and interraters score assessments 
blind to each others’ scoring. Training of scorers prior to scoring assessments is required, in the 
spirit of ensuring consistent knowledge of assessment scoring levels. The minimum acceptable 
level of scoring agreement is .80. This means particular assessment scores must be the same at 
least 80 percent of the time, in order for the score results to be reliable. If the reliability rate is 
less than .80 then the scores are not reliable. 

Reliability issues among the EPPs assessments were identified early in this study. These 
issues will be discussed in greater detail in the data collection and analysis portion of this paper. 

The New Quality Assurance System 

CAEP Standard 5.2 is especially relevant to the EPPs assessment validity and reliability. 
The component states, “The provider’s quality assurance system relies on relevant, verifiable, 
representative, cumulative and actionable measures, and produces empirical evidence that 
interpretations of data are valid and consistent” (CAEP Handbook, 2016). In other words, the 
assessment system is only as good as its data. If the data are valid and reliable then the needed 
foundation for analysis is in place. Then, these analyses lead to decisions driven by the intention 
of continuous improvement. 


Construct Validity Issues 
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Construct validity measures the extent to which an assessment measures what is intended 
to be measured. In the first semester of the study, the new division chair discovered construct 
validity issues. Students in their last semester of completing required course work, prior to their 
practicum and student teaching experiences, prepared to qualify for being admitted into “Stage 
2” of the program. Since the last accreditation visit, students prepared their electronic portfolios, 
and readied themselves for an interview driven by the state standards. 

Stage 2 committee members are comprised of full time Education faculty, local P-12 
district faculty or administrators, and Professors from other divisions across the college. On the 
way to the study’s first semester Stage 2 experience, the Investigator was told by faculty serving 
on a Stage 2 committee, from beyond the Education division, that they were sacrificing half of 
their week long fall break to review the electronic portfolios. They went on to say this would be 
the last time they would be doing so. Scheduling committee members to review students’ 
electronic portfolios during planned breaks dissuades them from being willing participants. 
Beyond dissuading committee members from serving on committees, using scheduled breaks for 
evaluating student portfolios can adversely affect scoring of them. Hard working faculty need 
scheduled breaks. Perhaps some committee members scored in haste, in an attempt to try to have 
at least a partial break from their usual workload. 

Once the interviews began a P-12 teacher who served on committees told the Investigator 
they had purposely ignored the emails sent by the Education division to serve on them. Since 
emails were being ignored, one of the Education division faculty members made a personal visit 
to the teacher’s school, located them, and secured their consent to serve on the committee. 
Among the concerns possessed by the P-12 teacher was the Stage 2 interview was entirely 
scripted, and they had been told they could not deviate from the state standards-driven script. 
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This same concern regarding the tight scripting of the Stage 2 interviews was subsequently 
expressed by an across the college Professor who had previously served on a Stage 2 committee. 
The metaphor they both independently used for the Stage 2 interview process was like being in a 
“straightjacket.” There were several concerns held by Stage 2 committee members beyond the 
education division. Additional concerns were directly related to the construct validity problem. 

During the first semester of the study, some of the students preparing for the Stage 2 
entry process did not take it seriously. The new division chair reviewed four of the electronic 
portfolios. The students were assigned to share their education philosophies from their early 
introduction to education course and a very recent education philosophy. In a pretest/posttest 
kind of way, these philosophy of education papers were supposed to reveal the pedagogical 
growth of the students across their programs. 

In all cases, the first and second philosophy of education papers were exactly the same, 
which of course would be measured as no growth across their several year programs. Students 
shared a letter of introduction in their electronic portfolios, as well. In one case a student 
described how they had always wanted to be a mathematics teacher, even though their program 
major was social science. Such carelessness abounded across the artifacts in the portfolios. 
Somewhere along the way these students came to conclude the portfolio process did not matter. 

Students’ videos were supposed to represent their best teaching of a lesson, used to judge 
if they were ready to student teach. In all cases reviewed by the new chair, the videos were the 
students’ first ones, from a teaching attempt earlier in their programs. The students’ unit plans of 
instructions were also no more than mundane and ordinary. All of the artifacts suggested the 
students were merely going through the motions in the portfolio process. The realities of the poor 
quality of some of the students’ Stage 2 electronic portfolios reinforced the complaints by 
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committee members, beyond the Education Division, about the hours of time it took to review 
the product. To compound the problems, in some cases portfolio reviewers invested many hours 
of work in reviewing careless student work. 

To compound the problems with the electronic portfolios, and substantial time invested in 
reviewing them by committee members, they were not the focus of the opportunity to interview 
the students for Stage 2 entry. Instead, the tightly scripted, state standards-driven interview was 
general in nature. Additionally, in most cases, groups of students were interviewed rather than 
being interviewed individually. The questions did not focus on the students’ two plus years of 
particular course product . There was a construct validity disconnect between the students’ 
electronic portfolio work and the “Stage 2” state standards interviews. 

Remedying the Construct Validity Issue 

Based on the feedback from Stage 2 committee members in semester one of the study, 
along with the Investigator’s conclusions regarding the questionable quality of the students’ 
electronic portfolio product, changes in the Stage 2 process were incorporated in the second 
semester of the study by the division chair. 

Study semester two changes in the Stage 2 process included: Assurance of quality of a 
second philosophy of education paper in the course where the paper occurred, the providing of 
the electronic portfolio earlier in students’ programs, rather than two weeks prior to the new 
presentations, a shift from committee members interviewing Stage 2 candidates driven by the 
state standards to Stage 2 candidates presenting their prescribed course product, within the a state 
standards framework, ending of the requirement for P-12 and across the college Professors to 
invest hours of time in reviewing the electronic portfolios, while at the same time providing them 
with access to them, changes in the schedules of when the electronic portfolios were to be 
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reviewed so they would not be during scheduled faculty breaks, freedom of committee members 
to ask any questions they like regarding the students’ presentations of their work, and of course 
needed changes in the EPP-created assessment instrument used to evaluate the student 
presentations rather than interviews. 

Some education faculty and students resisted the change from Stage 2 student interviews 
to student presentations. In one case a student broke down crying in the class where the 
portfolios were constructed, and was escorted to the education division chair by the education 
faculty member teaching the course. The student tearfully expressed how they had made many 
life sacrifices to pursue a teaching degree and career. They felt as though the change from the 
interview to a presentation of their work would cause them to fail, and they would not realize 
their dream of becoming a teacher. 

The division chair immediately expressed to the faculty member, in the presence of the 
student, that their support was needed in this change from interviews to presentations. The 
student’s concerns were heard by the chair. However, the chair dismissed them due to their not 
being substantive. The student was calmed down, and subsequently had a successful presentation 
experience. Buy-in by the Education faculty is critical to the success of change initiatives. 
Students look to them regarding how to respond to new program requirements. With changes in 
accreditation requirements, there must be changes in the teacher education programs. 

The New “Stage 2” Process to Ensure Construct Validity 

In division meetings leading to the Stage 2 changes, the chair requested that the Stage 2 
student presentations be made publicly available to the Education students. The chair used the 
metaphor of a music recital for what the student presentations could become. In other words, 
students seeking admittance to Stage 2 would share their best work, in presentation form, to the 
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committee and student audience. Education students observing the presentations would gain 
ideas for their future presentations. This student legacy approach seemed to have great potential 
for continuous improvement of the Stage 2 student presentations. 

Most of the faculty resisted the chair’s recommendation to cause the Stage 2 student 
presentations to be attended by fellow students in the programs. They felt the change from 
student interviews to presentations would be enough of a challenge for the students in the first 
semester. The chair consented and agreed to wait until fall semester, 2016 to make the Stage 2 
student presentations available to other Education students. 

The chair visited the class where the students were preparing for the Stage 2 presentations 
and discussed the new presentation approach. All students who subsequently presented in the 
new Stage 2 presentation model were successful. Feedback from committee members, especially 
those beyond the EPP faculty, was positive. It is likely the change from student interviews, with 
scripted questions not directly measuring student work, to student presentations with open-ended 
end of presentation question and answer opportunities will cause external committee members to 
be more willing to participate in future Stage 2 episodes. 

The change in Stage 2 from state standards -driven student interviews to state standards- 
driven student presentation addressed and remedied the construct validity problem that was 
obvious. The feedback from committee members, as well as the Investigator’s review of several 
electronic student portfolios, was the data analyzed leading to the decision for the needed 
changes. 

CAEP encourages EPPs to collect and analyze data, leading to changes with the intention 
of improvement. This example where the EPP addressed the Stage 2 construct validity problem 
should at least satisfy what is desired by the accreditor. 
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Reliability Issues 

Beyond issues with the EPPs instrument content, construct, and criterion validity, 
reliability problems with the assessment data became obvious, in the first semester of the study. 
As previously described, the EPPs unit-created assessments did not possess validity. There was 
at least one problem area with construct validity that was identified and remedied, and there were 
criterion validity issues due to the state’s program review practice incorporating required SPAs 
identification across course syllabi. Further, reliability issues compounded the validity problems. 

The assessment system, now known within CAEP Standards guidelines as the “Quality 
Assurance System,” relied on hard copy evaluation instruments being submitted to the EPPs data 
manager by committee chairs. For a variety of reasons, not all hard copy evaluations were 
collected, evaluated for scoring, entered into the database, and placed in students’ files. Without 
all of the needed completed evaluation instruments, the data manager could not compute 
assessment score averages. Additionally, the database did not have data cells in its design for 
both rater and interrater(s) scores. Since the database did not have the needed data cells to test for 
reliability, there was no possibility of calculating scoring agreement rates. Therefore, on more 
than one level, there was no reliability of scores for the EPPs assessments, along with no content 
validity for the EPP-created assessments, not to mention a hit and miss approach of ensuring the 
collection of all needed data. 

CAEP requires three cycles of data from the quality assurance system to be analyzed by 
the EPP, resulting in decisions regarding how best to continuously improve. Of course, if the 
data are neither valid or reliable then subsequent analyses and decisions are likely flawed. 


Life Happens 
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Beyond the validity and reliability issues possessed by the EPP, there were changes in the 
data manager position that impacted the assessment system. In spring semester, 2015 the data 
manager suddenly died. Data entry for the spring semester of 2014-2015, was subsequently non- 
existent until spring semester of the 2015-2016 academic study year. During the study year the 
data manager who had been hired in June, 2015 departed in April, 2016. This pushed data entry 
for 2015-2016 into the 2016-2017 academic year. A third data manager began near the end of 
this one year study, academic year 2015-2016. 

Such turnover in the high stakes position of EPP data manager position affects the 
ability to collect and analyze data. Compounding the data manager turnover issue, the college 
completed a search for a division chair who was supposed to begin work in academic year 2014- 
2015. Unfortunately, the person who accepted the chair position fell ill and was unable to come 
to the college. This resulted in the division losing a year of leadership in accreditation 
preparation work. A new chair was hired July 1, 2015. 

Faculty Workloads Impacting Assessment Scoring 

Beyond the hard copy data collection problem, the missing database data cells problem, 
the data manager turnover problem, and the new chair being unable to come to the college, the 
Education faculty, overwhelmed by workloads, often had to score assessment instruments in 
haste, including during scheduled breaks. 

The scoring of assessments simply had to be subordinated, in order for faculty to be 
successful in their many tasks. As mentioned previously, faculty college breaks had been used 
for evaluating electronic portfolios. This calendar design, where needed college breaks were 
being used for education faculty, and others’, assessment of student product, further 
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compromised the assessment instrument scoring. Faculty being time -pinched, due to their 
extraordinary workloads, added to the scoring reliability issues. 

Fall, 2017 CAEP Visits 

CAEP requires three cycles of assessment data. The EPP will have three cycles of data, 
but due to the many issues cited they will be suspect. For EPPs having CAEP accreditation visits 
in Fall, 2017, it is expected the new quality assurance system be in place and operational. 
However, CAEP does not necessarily expect the three cycles of data used in the accreditation 
visit to be reflective of the new standards requirements. 

Instead, verbiage regarding the shortcomings of the former system are to be identified 
and shared, along with descriptions of the new system. The quality assurance system that aligns 
with the new CAEP standards must be operational at the time of the self-study, one year prior to 
the accreditation visit. Typically CAEP self-studies are due eight months prior to the 
accreditation visit. 

Electronic Portfolios 

In the first semester of the two semester study, students received their electronic 
portfolios two weeks prior to their Stage 2 interviews. Many artifacts were required to be placed 
in the electronic portfolio in a very short time. Items from across the students’ three years of 
courses were expected to be placed in them. In some cases the items had been misplaced by the 
students across their program journeys. Due to construct validity issues previously discussed, 
much of the work expended by students on the portfolios seemed futile. What was being 
measured in the interviews did not line up with the years of student work. 

Several of the programs have academic majors outside of the EPP. Obviously, students in 
those majors have far fewer courses within the EPP. The higher level courses they do have 
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within the EPP are “back-ended” in the programs, meaning their high stakes courses where 
artifacts are explained and produced are taken in a relatively short period of time, near the end of 
the courses in their programs. Compounding the tight time window, and intensive creation of the 
portfolio artifacts, was access to the Stage 2 two weeks prior to the student interviews. The short 
time in which students had to create their electronic portfolios exacerbated the problems with the 
intensity of producing the required artifacts. Further, students in these several programs, with 
many courses outside of the EPP, often did not have all artifacts they were supposed to have, 
when coming into the class where the portfolios were built, for a variety of reasons. 

Additionally, students from all programs are placed in one class, consisting of two 
sections, where the portfolios are finalized. Students with majors in the EPP are at a natural 
advantage when it comes to having the needed product for the portfolios, in that they have more 
courses across their academic programs within Education, and greater guidance, where the 
required artifacts can be created. Receiving the electronic portfolio two weeks prior to their 
Stage 2 interviews caused considerable anxiety for many of the students, especially those whose 
majors were outside of the EPP. 

Changes in When Students Receive Their Electronic Portfolios 

In response to these data, and its analysis, among the changes implemented during the 
second semester of the academic year study was the providing of the electronic portfolios to 
students earlier in their programs. While the earlier providing of the electronic portfolios to 
students did not address the immediate problem the current Stage 2 candidates faced, its intent 
was to reduce the students’ problems and anxieties in future semesters and years. 

Education students are advised to take the Technology in Education course early in their 
programs. The course was a logical place to provide students with their needed electronic 
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portfolios. Students receiving their electronic portfolios early in their programs have the potential 
to cause them to be more likely to place the required artifacts from courses along the way. 

Placing their artifacts in the electronic portfolio across the courses in their programs may also 
reduce the problem of lost documents. It may also reduce the problems students with majors 
outside of the EPP have experienced with the electronic portfolios. The instructor of the 
technology in education course welcomed the opportunity to guide students through their 
accessing and creating the electronic portfolios. 

Students’ Videos of Teaching Episodes 

One of the items required in the electronic portfolio is a video of the student’s best 
teaching episode. In the first semester of the study, an online software application for video 
viewing was sent to “Stage 2” committee members. They were instructed to download the video 
software so they could then view students’ video teaching episodes. There were many problems 
with the video software. Some browsers were not compatible with it. The problems with 
downloading the needed software, for which to view the student videos, hindered the viewing of 
them. It also added to the aforementioned frustrations of the time needed to review the electronic 
portfolios by committee members. 

The students with majors outside of the EPP typically had only one video from which to 
choose, for their best video performance. The course in which the final touches were made on the 
electronic portfolio required a video episode. However, the deadline for turning in the teaching 
episode video in the course was one week after the electronic portfolios were due. This caused 
many students, especially those with academic majors beyond the EPP, to not have at least a 
second video from which to choose for their portfolios. 
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While all students should have understood the need for another video of their teaching, 
the fact that the deadline for it being a week after the electronic portfolio was due caused some to 
not take the needed time to do them. Instead, students focused on the demands of completely 
preparing their electronic portfolios in the mere two weeks provided, not to mention other course 
requirements and demands in other courses. This resulted in many students’ first attempts at 
teaching a lesson to be the video selected by them that was supposed to be the video of their best 
teaching performance. First attempt teaching episode videos being shared by students as their 
purported “best video teaching episode” was another construct validity issue. 

First attempts at teaching are typically not “best” episodes, resulting in product that does not best 
measure what is intended to be measured. 

CAEP expects student product to be assessed in valid and reliable ways. In order for this 
to occur, the students must have the needed time and opportunities to carefully produce it. 
Additionally, the EPP faculty and external review committee members must have the needed 
time and opportunities for carefully evaluating it. The product should reflect the students’ best 
attempts to satisfy the requirements and standards. 

In future semesters, there will be an increase of clinical observations of students. These 
formalized clinical observation opportunities will provide students with more opportunities to 
video practice teaching episodes, resulting in better product of best teaching episodes for their 
electronic portfolios. Further discussion regarding the increased supervision of clinical 
experiences shall be shared in the next portion of this paper. 

Clinical Experiences 

While it is common for nationally accredited EPPs to have clinically observed teaching 
episodes prior to student teaching, there were none evident during the first year of this study. 
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Also, there was not a common assessment used by college supervisors and cooperating teachers 
during student teaching, not to mention prior to student teaching. Reliability could not be 
measured in the clinical observations. 

In the continuous assessment design that had been approved four years prior to this study, 
during the national accreditation process, the EPP agreed to have two teaching episodes 
clinically supervised prior to student teaching. These two supervised teaching episodes had been 
abandoned and were not apparent during the study. 

The new chair of the division, during the year of this study, discovered the prior plan and 
inquired to faculty as to why the pre-student teaching episodes were not occurring. The faculty 
told the new chair the p-12 school teachers did not like it so the approved plan was discarded. 

The chair asked the faculty the reason why there was not a common observation 
instrument used during student teaching. The faculty reported the co-teaching model does not 
allow for the cooperating teacher to observe the student teacher since “they are always teaching 
together.” This, per the faculty, resulted in the cooperating teacher needing to use a different 
assessment instrument than the college supervisor who visits the class to observe the student 
teacher. 

The new chair suspected the faculty did not understand the co-teaching model, in that 
student teachers do in fact teach on their own during the experience. Or, the faculty was 
misrepresenting the definition of co-teaching in order to attempt to avoid needed changes. 

For clarity regarding the co-teaching model, the chair followed up with the state 
professional standards board. The board representative stated the faculty’s interpretation of the 
co-teaching model was a misinterpretation of the regulation. Student teachers do in fact have 
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standalone teaching experiences, including their teaching of a unit plan of instruction. The new 
chair shared this information with the faculty. 

Since cooperating teachers can observe their student teachers, they can in fact use the 
same observation instrument used by the college supervisors. Naturally, if both the college 
supervisor and the cooperating teacher use the same assessment instrument scoring reliability can 
be measured. 

Formerly, the opportunity to measure for scoring reliability did not exist, since the 
observers were using different assessment instruments. But, now the same clinical assessment 
instrument can be used in student teaching, and scoring reliability can be measured. The 
assessment being used is the state’s assessment instrument for P-12 teaching. This causes it to be 
a pre-validated instrument. Still, within the guidelines of new CAEP Standard 2, there should be 
consensus between the P-12 partner districts and the EPP regarding the instrument’s use. 

Having no mutually observed teaching episodes by cooperating teachers and college 
supervisors, using the same assessment instruments, prior to student teaching is a problem for the 
EPP. The other EPP in the state preparing for a fall, 2017 accreditation currently has three 
teaching episodes in their programs prior to student teaching, observed by both cooperating 
teachers and college supervisors, using a common observation instrument. This coming 
academic year, 2016-2017, the number of supervised teaching episodes orchestrated by the other 
EPP, prior to student teaching, is increasing from three to six. It is critical the EPP have at least 
the previously accredited-approved two observed teaching episodes prior to student teaching. 
Two observed teaching episodes, prior to student teaching, are to start in the 2016-2017 


academic year. 
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Abbreviated versions of the pre-validated state observation instruments are planned to be 
used in the two episodes. The EPPs reconstituted teacher education committee, informed by new 
CAEP standard component 5.5, will consider the versions for recommended use at their August, 
2016 meeting. 

New CAEP standard 2, focused on P-12/EPP partnerships, including clinical practice, 
and an abridged version of it created by the EPP chair, is being shared with partner school 
district leaders beyond this study, during the summer of 2016. Much must be done to reconfigure 
the EPP/P-12 partnerships in a short time, so they are enhanced and meet new CAEP Standard 2. 

P-12 School Partnerships 

CAEP Standard 2 is focused on P-12/EPP partnerships (CAEP Handbook, 2016). Beyond 
students’ clinical experiences, CAEP standard 2, comprised of three components, includes such 
things as the P-12/EPP shared responsibility model, P-12/EPP co-construction of observational 
instruments and clinical experiences, EPP-provided required online clinical training availability, 
and analyses of P-12/EPP partnership clinical data. 

Standard 2.1 requires P-12 school/EPP shared responsibility for continuous improvement 
of candidate preparation. Evidence for 2.1 include a description of partnerships (MOU), along 
with documentation that the partnership is being implemented as described. 

Artifacts include: schedules of joint meetings (in the EPPs case TEC meetings, and other partner 
meetings as needed, documentation of stakeholder involvement, a shared responsibility model, 
technology-based collaborations, evidence of co-constructed observational instruments (content 
validity exercise), and evidence of co-constructed candidate expectations during clinical 
experiences. 
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Standard 2.2 measures: Evidence of co-selection of College Supervisors (CSs) and 
Cooperating Teachers (CTs), evidence of criteria for selection of clinical educators, including 
recent field experience and currency in relevant research. Resources available online: clinical 
educator orientation available both in person and online, performance evaluations of CSs, CTs, 
and candidates, collected data used to modify clinical experiences, records of remediation and/or 
counseling out of candidates available. 

Standard 2.3 measures: Analyses of data for what works and what doesn’t in an ongoing 
manner, cross referencing findings and conclusions, and continuous answering to the research 
question: What is it about the clinical experiences (that is, depth, breadth, diversity, coherence, 
and duration) that can be associated with the observed outcomes? 

CAEP Standard 5 and the Teacher Education Committee (TEC) 

New CAEP Standard component 5.5 demands a reconfiguration of the committee, or 
council, which reviews EPP recommendations, etc. on a regularly scheduled basis. The 
committee should be comprised of P-12 District/School/Clinical Education Leaders, P-12 
Educators, Recent Graduates (Alumni) of Education Division Teaching in P-12 Schools, Current 
Education Students, Full Time Education Division Faculty, EPP Program Coordinators, an 
Education Division Adjunct Representative, and, perhaps, an Ex-Officio College Administration 
Member which is optional. 

During the study year, the EPPs Teacher Education Committee did not reflect the CAEP 
prescribed membership. Attendance by members beyond the EPP was low, or not apparent. The 
committee met on the Thursday of the first week of the two semesters. By then, students who 
were to be considered for Stage 1 admission into the programs were already in the restricted 
professional courses requiring admission. Student teachers who were to be considered for 
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recommendation were already placed in P-12 school settings. Thus, the recommendations on the 
part of the committee were little more than ceremonial. Beyond the time of this study, the EPP 
needs to reconsider the timing of the meetings so the recommendations have substance, in 
addition to reconstituting its membership. 

College Wide Recruiting Efforts 

CAEP Standard 3 is comprised of six components. Standard 3.1 is focused on college 
wide recruiting efforts. It requires a five year college wide recruitment plan, with annual detailed 
plans for each of the five years. It also requires annual reporting of the success of the recruiting 
efforts. 

The goal of the new recruitment plan is to recruit students for the teacher education 
programs that “reflect the diversity of America’s P-12 students” (CAEP Handbook, 2016). It 
should be noted the P-12 student population in America became a majority minority in August, 
2014. 

Emphases on Science education, Math education, English Language Learner (ELL) 
education, and Special Education are CAEP targeted programs for diverse student recruiting. 

The chair of the EPP plans to share the new, required five year college-wide recruiting plan with 
the dean of admissions, president’s office, and vice president of academic affairs during the 
summer of 2016. This should provide time for the college to create the needed five year 
recruitment plan, in order for it to be activated in academic year 2016-2017, one year prior to the 
CAEP visit. It should also provide the EPP the opportunity to write about the new recruitment 
plan in its accreditation self study that will be written during the 2016-2017 academic year. 


State Partnerships 
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CAEP standard 5 describes the quality assurance system. It measures the performance of 
CAEP standards one through four. There are eight annual reporting measures as defined in the 
EPPs annual report, and in CAEP state partnership agreements. Of the eight measures, four are 
known as impact measures and four are outcome measures. Of the four impact measures, number 
three is known as “Employer satisfaction and completer persistence” (CAEP Handbook, March, 
2016). The data for this impact measure is in large part state derived, and depends on the state’s 
collection of it. Among the four identified four outcome measures, numbers seven and eight are 
dependent on state collected data. Measure seven is employment rate, measuring the EPPs 
success in graduates gaining teaching jobs, and staying in them. Measure eight is consumer 
information. This measure possesses an asterisk in the CAEP handbook, in that it is not used in 
the accreditation decision. 

The EPPs state partner consistently provides needed support and timely guidance. The 
EPP chair works closely with state professional standards board representatives. The chair and 
state representatives held a meeting in the first month of the new EPP chair’s stint, in the spirit of 
meeting and discussing the upcoming accreditation requirements. The state provided seminars 
during the study year to receive input and feedback, regarding the construction of the new 
CAEP-driven data gathering system. The EPP chair attended several meetings to be apprised of 
the state’s work and CAEP partnership. 

Quality Assurance System Revisions 

CAEP standard 5, comprised of five components, describes the new quality assurance 
system. Previously, these data collection designs were known as continuous assessment systems 


but CAEP has labeled them with this new moniker. 



CAEP CHALLENGES FOR A MID-SOUTH U.S. COLLEGE 


24 


CAEP standards one through four outcomes are continuously measured by standard five, 
the quality assurance system. These data are monitored by the CAEP eight annual reporting 
measures (CAEP Handbook, March, 2016, p. 63). As previously explained, EPPs with 
accreditation visits in fall, 2017 are not necessarily expected to have new CAEP aligned data 
from previous data cycles. However, EPPs with these first accreditation visits are expected to 
have their quality assurance systems in place, and operational, at the time of the creation of their 
self studies, one year prior to their visits. 

CAEP expects EPPs to have three cycles of data that have been collected, analyzed, and 
used for decisions for continuous improvement. The cycles can be either semesters or academic 
years, which is something new in teacher education cycle reporting. Previously, cycles could 
only be entire academic years. 

The EPP in this study does in fact have three cycles of data. However, there are many 
issues with these data, which will be shared in program reports and the CAEP self study. The 
Investigator discovered EPP created assessment instruments have not been tested for validity. 
Therefore, at the time of this study they did not possess validity, as also cited as an area for 
improvement by a visiting accreditation team. 

It was also discovered there was no reliability in scores of assessment and observation 
instruments. Further, there were no opportunities for testing reliability, statistically. While these 
items should, and do, elicit substantial concern, they provide opportunities for EPP writing in the 
self study regarding its baseline, and plans for improvement. 

Conclusion: Preparing for the CAEP visit 

EPPs who are preparing for CAEP accreditation visits in fall, 2017 and beyond face 
many challenges. CAEP standards substantially raise the proverbial accreditation bar. Changes in 
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required pre-service candidate recruiting, partnerships with P-12 school district partners, state 
partnerships, and the new quality assurance system are among the new challenges. 

While CAEP insists EPP created assessments should be sent to them for review and 
feedback, EPPs facing fall, 2017 accreditation visits will not be enjoying such reviews and 
feedback. CAEP states there is not enough time for the reviews. However, the EPPs will still be 
held to the same instrument standards as EPPs who will experience later CAEP visits, and the 
needed instrument evaluations and feedback. With this in mind, it is best to locate and implement 
rubrics for assessment that are pre-validated. 

The EPP studied by the Investigator did not seem to understand the need for substantive 
change during the academic year studied. A new chair began in the position during July, 2015. 
This chair was the thirteenth one for the division in 26 years. Obviously, there was considerable 
leadership turnover in the EPPs recent history. 

Compounding this problem, three fourths of the full time education faculty was new to 
the EPP in the study year. Across the past ten years, the number of full time education faculty 
decreased by forty percent. A considerable portion of the professional courses across the 
programs are taught by adjunct faculty. 

The new chair’s effort to cause the Stage 2 interview to be changed to a Stage 2 
presentation met faculty resistance. The chair’s effort to incorporate the previously state- 
approved two clinical observations of students prior to student teaching met considerable faculty 
resistance as well. Most, and in one case all, external members of the teacher education 
committee, beyond full time education faculty and some adjuncts, did not attend the two high 
stakes TEC meetings across the academic year. 
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During the one year study, the new EPP chair came to understand the particular 
challenges of the division, as well as the new challenges of CAEP accrediation. Change is 
difficult, and often resisted. The new CAEP accreditation requirements are daunting. Each EPP 
is different and unique. However, they all possess internal challenges that can impact CAEP 
success. Every EPP also possesses merit that can help them with CAEP accreditation success. 

The EPPs who are first to experience the new CAEP accreditation preparation and visits 
in fall, 2017 have great potential for providing others with insights regarding what can be 
expected. It is the Investigator’s hope this case study can be useful for the EPP studied, as well as 
for other teacher education programs, in their contemplations about how best to prepare for the 
challenges of CAEP accreditation. 

States are now required to rank teacher education programs. These rankings will 
exacerbate the reduction of traditional higher education teacher education programs. Programs 
ranked at the bottom in each state will naturally disappear. 

EPPs and their parent colleges will need to invest considerable time and monies in their 
programs, in order to continue to be successful. Teacher education programs have been under 
siege since 2010, and the requirements for their being successful are only becoming more 
difficult (Moffett, 2012). 

For the many teacher education programs that will close or fail in the next few years, new 
federal regulations provide special “provisions” for “academies” that will take their place. “Title 
2 dollars can now be used to support the expansion of entrepreneurial “start-up programs” (i.e., 
teacher preparation “academies”) (New ESSA Regulations, Washington Post article, October 24, 
2016). 

Recommendation for Further Study 
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CAEPs Standard 2, focused on enhanced P- 12/College partnerships, requires substantial, 
and continuous, research. The Investigator recommends EPPs devote a considerable portion, if 
not all, of their research energies to this portion of the new standards. 

EPPs must partner with P-12s to generate the needed, new research. Such research 
provides opportunities for needed faculty annual scholarly productivity. In the case of the EPP 
studied, the new chair intends to lead the new Standard 2 research endeavors. Research 
opportunities for Investigators, conducting case studies of teacher education programs, are 
evident in this area. 
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